Successful and sustained implementation of a behaviour-change informed strategy for emergency nurses: a multicentre implementation evaluation

Background Implementing evidence that changes practice in emergency departments (EDs) is notoriously difficult due to well-established barriers including high levels of uncertainty arising from undifferentiated nature of ED patients, resource shortages, workload unpredictability, high staff turnover, and a constantly changing environment. We developed and implemented a behaviour-change informed strategy to mitigate these barriers for a clinical trial to implement the evidence-based emergency nursing framework HIRAID® (History including Infection risk, Red flags, Assessment, Interventions, Diagnostics, communication, and reassessment) to reduce clinical variation, and increase safety and quality of emergency nursing care. Aim To evaluate the behaviour-change-informed HIRAID® implementation strategy on reach, effectiveness, adoption, quality (dose, fidelity) and maintenance (sustainability). Methods An effectiveness-implementation hybrid design including a step–wedge cluster randomised control trial (SW-cRCT) was used to implement HIRAID® with 1300 + emergency nurses across 29 Australian rural, regional, and metropolitan EDs. Evaluation of our behaviour-change informed strategy was informed by the RE-AIM Scoring Instrument and measured using data from (i) a post HIRAID® implementation emergency nurse survey, (ii) HIRAID® Instructor surveys, and (iii) twelve-week and 6-month documentation audits. Quantitative data were analysed using descriptive statistics to determine the level of each component of RE-AIM achieved. Qualitative data were analysed using content analysis and used to understand the ‘how’ and ‘why’ of quantitative results. Results HIRAID® was implemented in all 29 EDs, with 145 nurses undertaking instructor training and 1123 (82%) completing all four components of provider training at 12 weeks post-implementation. Modifications to the behaviour-change informed strategy were minimal. The strategy was largely used as intended with 100% dose and very high fidelity. We achieved extremely high individual sustainability (95% use of HIRAID® documentation templates) at 6 months and 100% setting sustainability at 3 years. Conclusion The behaviour-change informed strategy for the emergency nursing framework HIRAID® in rural, regional, and metropolitan Australia was highly successful with extremely high reach and adoption, dose, fidelity, individual and setting sustainability across substantially variable clinical contexts. Trial registration ANZCTR, ACTRN12621001456842. Registered 25 October 2021. Supplementary Information The online version contains supplementary material available at 10.1186/s13012-024-01383-7.


Contributions to the literature
• Evaluating methods to translate evidence effectively and rapidly to emergency practice is crucial to keep pace with increasing demand for emergency care.Strategies to overcome implementation barriers improve clinical trial performance and intervention implementation in the complex emergency setting.• Our behaviour-change strategy enabled maximum, sustained uptake of an intervention in a variety of emergency settings despite the COVID-19 pandemic and catastrophic flooding.• The Mechanisms of Action "Optimism" and "Social Professional Role and Identity" addressed barriers related to feelings of a lack of support by management and a lack of time to change practice, previously reported as having no definite links.

Background
As demand for emergency care increases globally so too does the need for timely recognition and treatment of the acutely ill and injured for high quality and safe healthcare [1].Emergency nurses are the first emergency department (ED) clinicians to assess and manage patients, and their practice is fundamental to patient safety.Emergency nurses are responsible for the initial and ongoing assessment, management, and safety of patients of all ages, with varying degrees of clinical urgency and severity.To enable safe and quality emergency nursing care, HIRAID ® was developed [History including Infection risk, Red flags, Assessment, Interventions, Diagnostics, communication, and reassessment)] [2-4] (Fig. 1).HIRAID ® is an emergency nursing framework used by emergency nurses for any patient presentation and the only validated framework designed to enable emergency nurses to systematically assess and manage ED patients [5].In feasibility, efficacy and replicability studies, use of HIRAID ® reduced clinical deterioration related to emergency nursing care, improved clinical documentation [6], handover [3] and reduced treatment costs [7].
Successful and sustained implementation of any intervention in health care is challenging [8].Many large trials have failed to effectively change emergency nursing practice [9,10].EDs have well-established barriers of high levels of uncertainty arising from the undifferentiated nature of patients, chronic resource shortages and limitations, workload unpredictability, high staff turnover and constantly changing environments.They have unique external (policies, resources, and incentives) and internal (local organizational readiness, infrastructures, and workflows) [11] contextual characteristics, including micropolitics [12], that influence implementation.Early planning including the identification of particular ED characteristics and associated factors that influence clinician behaviour is necessary for the successful implementation of an intervention [13].Strategies to achieve successful clinician behaviour change, clinical trial performance, and intervention implementation in complex healthcare settings such as EDs can be successful with behavioural optimisation [14][15][16][17].Put simply, evaluating what works in the emergency context is critical to future implementation.
To implement HIRAID ® in 29 Australian rural, regional and metropolitan EDs as part of an effectiveness-implementation hybrid study [18], we developed and executed a behaviour-change implementation strategy to mitigate the above barriers using a systematic process guided by the behaviour change wheel and theoretical domains framework [19].Developed with end users, implementation experts and stakeholders a multifaceted implementation strategy was developed with 21 behaviour change techniques (BCT) from the behaviour change technique taxonomy (BCTT) [20].Strategies included an educational program-HIRAID ® Instructor (train-the-trainer) and Provider courses including eLearning module, and face-to-face education; visual prompts in the electronic medical records such as a template to document the HIRAID ® assessment; local change champions to support, model in clinical practice; a video featuring clinical leaders and individualised audit and feedback.More detail is provided with the results and elsewhere (Curtis K, Kennedy B, Considine J, Murphy M, Kourouche S, Lam MK, Shaban RZ, Aggar C, Hughes JA, Fry M: Design of a data-driven implementation strategy to optimise clinician behaviour change at scale in complex clinical environments: A multicentre emergency care study, under review).
Prior to analyses to measure the effectiveness of the intervention on patient and health services outcomes, the effectiveness of the implementation strategy required evaluation, particularly around implementation fidelity, which is the extent to which an intervention is delivered per the intended implementation plan [21,22].Fidelity is critical to the internal and external validity of implementation [22].Implementation science theories and models such as the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation (dose/fidelity) and Maintenance / Sustainability) provide a robust framework for evaluating implementation outcomes [23].Improved implementation outcomes can directly contribute to better patient outcomes [24].The aim of this study was to evaluate the behaviour-change informed HIRAID ® implementation strategy on reach, effectiveness, adoption, quality (dose, fidelity) and maintenance (sustainability).

Methods
We used a type II effectiveness-implementation hybrid design [25] including a step-wedge cluster randomised control trial (SW-cRCT) to evaluate: i) the patient outcomes of HIRAID ® implementation at scale (effectiveness) and ii) strategies that enable optimal HIRAID ® uptake (implementation).The hybrid approach allows testing of the implementation strategy at the same time as observing the outcomes of the intervention [26].The purpose of this paper is evaluation of the implementation strategy, the patient outcomes and cost effectiveness will be reported separately to allow for a more comprehensive report.The HIRAID ® implementation strategy evaluation plan was developed during study design guided by the RE-AIM framework [27] and Scoring Instrument [28] (Table 1).
Reach was measured by the proportion of eligible nurses participating in the program, effectiveness through the end user (nurses) opinions on the usefulness of HIRAID ® , and adoption by the proportion of eligible sites implementing HIRAID ® .Dose was the consistency of exposure to HIRAID ® across settings and fidelity, the degree to which individual components and the overarching implementation strategy was delivered.Sustainability was determined through demonstration of ongoing   1).We used multiple methods to achieve this as recommended by Glasgow et al. [27].Also guided by suggestions from the behaviour change wheel the BCT taxonomies was used to guide the evaluation plans for implementation fidelity.Quantitative data were collected to measure proportions and determine the level of each component of RE-AIM achieved.Qualitative data were used to understand the "how" and "why" of quantitative results, and any deviations or adaptations to the proposed implementation strategy.

Implementation and settings
The HIRAID ® trial was a SW-cRCT across 29 EDs from two Australian States with over 1300 emergency nurses.The 29 EDs were grouped into four clusters per their local health district grouping (Table 2).Each cluster was randomised, commenced the trial in the control condition and sequentially crossed over to the intervention condition: the first cluster commenced November 9th, 2020.The "go live" comprised commencing HIRAID ® training for emergency nurses, availability of the HIRAID ® documentation templates, and formal instruction from hospital executives that HIRAID ® was starting to be used.The clusters represent a range of geographically and clinically diverse settings, from large teaching hospitals to some rural services without medical services onsite, with nursing staff required to call them in as required and/or use telehealth.The implementation strategy and development are described in detail elsewhere (Curtis K, Kennedy B, Considine J, Murphy M, Kourouche S, Lam MK, Shaban RZ, Aggar C, Hughes JA, Fry M: Design of a data-driven implementation strategy to optimise clinician behaviour change at scale in complex clinical environments: A multicentre emergency care study, under review) (Supplementary file 1).

Participants
There were two groups of participants in this study: permanently employed emergency nurses (HIRAID ® providers), and HIRAID ® Instructors.All permanently employed emergency nurses were eligible and encouraged to undertake HIRAID ® Provider training to adopt HIRAID ® in their clinical practice.HIRAID ® Instructors were appointed at each site to aid in the consistent application and fidelity, most of whom were part of the existing nursing workforce at the cluster and in a position to support the change as part of their roles, such as the cluster Clinical Nurse Educators (CNEs) and Clinical Nurse Consultants (CNCs).Several of the smaller sites in NSW did not have a CNE, so we recruited a HIRAID ® Instructor in consultation with site managers.Site HIRAID ® Instructors were invited to attend weekly implementation team meetings and were asked about any implementation complications or adjustments.

Data sources and collection procedures
There were three primary data sources: (i) a six-month post HIRAID ® implementation emergency nurse survey; (ii) HIRAID ® Instructor surveys; and (iii) six-and twelve-week documentation audits.Data from each source were mapped and integrated to RE-AIM outcomes per Table 2. Information from the project timeline as registered on the clinical trials website (ACTRN12621001456842) was also used [34].

Data source 1: Emergency nurse survey
An electronic cross-sectional survey of all permanently employed emergency nurses was conducted to ascertain their perspectives on the usability and implementation of HIRAID ® six months post implementation "go live" at each ED.Casual staff were not approached as they were generally not included in the training program and a transient workforce.The first section of the survey collected participant characteristics (place of work and years of nursing experience).This was followed by 10 multiple-choice questions to ascertain their perspective on strategies used to introduce HIRAID ® to their ED.For example, "Do you use the HIRAID ® documentation templates in ED for your initial documentation (post triage)?If not, why not?".The questions in the survey were used to identify if staff used the implementation strategies (reach), how often they were used (dose), and about how the strategies were delivered (adaptations) and were adapted from a previous survey [35].All questions had free text options for participants to explain their response (Supplementary material 2).Survey data were collected and managed using RED-Cap ™ (Research Electronic Data Capture) a secure, webbased software platform and electronic data capture tool hosted at the University of Sydney [36].The survey was tested for face validity during a pilot phase and minor changes to formatting and flow were made with testing by the research team and key stakeholders at the sites.Surveys, with a participant information and consent form clearly stating the voluntary nature of the survey and anonymity of survey responses, were distributed to all eligible staff at each cluster by departmental managers and kept open for four weeks.Reminders were sent weekly.An incentive was offered for survey completion (a variety of $20 AUD vouchers suitable for participant's location, for example a local cafe) as a key strategy known to increase response rates [37].

Data source 2: HIRAID ® Instructor surveys
There were two HIRAID ® Instructor surveys to identify the dose and fidelity of the implementation strategies.Firstly, a five-question survey was distributed at the completion of the Instructor course to determine if participants felt they had a good understanding of what was expected of them as a HIRAID ® Instructor and that they were confident in their ability to deliver HIRAID ® training using a 5-point Likert scale (Often/ Always, Sometimes, Rarely, Never, not sure).A second electronic survey was distributed at 12 weeks post-implementation asking to what degree the 21 behaviour change techniques were implemented [22] (not used at all, rarely used, sometimes used and used most or all the time).
Adaptations to the way the education program was delivered was also scored 1 (nil changes) to 3 (major changes from plan) (Supplementary material 1) Understanding the adaptations was important in evaluating the strategies relevant to the contexts.
Following testing for face validity, these survey data were also managed using REDCap ™ [36].Reminders were sent weekly for 4 weeks.An incentive was offered to participants on survey completion (30AUD supermarket voucher) [37].

Data source 3: Documentation audits
At six, 12 and 24 weeks post "go live" at each site, a documentation audit was conducted by the onsite research nurses to determine if nurses were using HIRAID ® documentation templates to record their initial patient assessment.Three consecutive initial nursing patient assessments in the medical record, from every nurse rostered on any shift during a seven-day period, were reviewed at all sites to determine if nurses used the HIRAID ® template to record their initial nursing assessment, and, that the correct template was used (for example, paediatric template for a paediatric patient).The reports were distributed to the lead investigators at each site for feedback.

Data analysis
Quantitative data were extracted and imported into Statistics Package for Social Sciences (SPSS) Version 22.0 [38] for analyses.Numeric data was analysed descriptively using median and interquartile range (IQR) due to the skewed distribution, while categorical data was analysed descriptively using counts and percentages.Kruskal-Wallis test was used to examine the overall relationship between experience and site, with Mann-Whitney test was used for pairwise comparisons.Chi-square test was used to examine the association between position and qualification with site.Overall significance set at 0.05.Bonferroni adjustment applied to multiple comparisons.Pairwise comparisons was significant at 0.008.
Qualitative data from surveys, were imported into NVivo ™ v14 [39] and analysed collectively using the Graneheim [40] method of inductive content analysis by one author (KC).The text for each of the questions was condensed and coded into meaning units then grouped codes with similarities together into sub-themes [41].Abstraction and interpretation of the latent content was kept low as the content was brief.Broader themes were grouped from the sub-themes considering the meanings.Findings from the quantitative and qualitative components were merged in a table to simultaneously array the quantitative and quantitative results using a convergent mixed methods approach [42] Implementation fidelity was assessed by examining percentage of agreement calculated from the HIRAID ® Instructor survey that captured the degree to which each implementation strategy and Behaviour change techniques (BCTs) were delivered as intended [20].Although there is no actual agreement to what comprises an acceptable percentage of agreement [43], high fidelity may be represented by a percentage of agreement of 80-100% and low integrity 50% or less [44].We selected a goal of 80% or more of Instructors indicating that the intervention was delivered as intended (i.e. a score of 4) to indicate high fidelity, 60-80% moderate delivery, and 59% or less for low fidelity [45].Percentage of agreement was calculated, where possible to evaluate setting and individual results.Subgroup comparisons between clusters were conducted.

Results
All 29 sites agreed to participate and adopt HIRAID ® , held on site training, and allocated non clinical time for their 1377 nurses to complete HIRAID ® Provider training.Emergency nurses (39% n = 534) completed the six-month post-implementation survey and had a median(IQR) 4(1-11) years ED experience.Respondents from the two rural regional clusters were more experienced with a median(IQR) 5 (2-13) and 8 (2-15) years emergency nursing) compared to the metropolitan clusters who had a median(IQR) of 2 (1-5) and 3 (1-10) years emergency nursing experience (KW(df3) = 49.7;p < 0.0001).More rural regional respondents had post graduate nursing qualifications (63.9% and 71.3% vs 43% and 58.1%, (X2(df9) = 27.7,p = 0.001).Over half the designated Instructors [64% (n = 90/145)] completed the Instructor survey, representing all clusters, although this proportion is likely closer to 70% as several instructors left the ED or changed roles during the implementation as noted in the implementation log managed by the lead investigator for each cluster (Table 3).

Reach
The setting and individual reach of the intervention was high.At least one senior nurse from each site completed Instructor program equating to 100% setting reach.Individual reach was also high.At the end of the 12 week implementation period, 91% (n = 1255) of eligible nurses had completed at least one aspect of the HIRAID ® training program.The majority of nurses [82% (n = 1123)], representing all 29 sites had completed the full HIRAID ®  training program.This increased to 96% (n = 1323) at 6 months.A larger proportion of eligible nurses from the metropolitan clusters completed the training compared to those from the rural / regional clusters.Data were not collected on the characteristics of participants to enable comparison with nonparticipants (Table 4).

Effectiveness
Of the staff who completed the post implementation survey, most (78.1%, n = 419) felt HIRAID ® helped to "teach new emergency nurses how to assess and manage patients".This fell short of our target > 80% of positive participant opinion on the usefulness of HIRAID ® .There was no significant variance across clusters.More than two thirds [72.1% (n = 385)] reported they thought about the HIRAID ® steps as they do a patient assessment most or all the time.More than two thirds (68.5%, n = 366) felt HIRAID ® helped them remember to do all parts of a full assessment when needed.The metropolitan cluster with the youngest and least clinically experienced workforce had the largest proportion of those reporting (86.3%, n = 138, p < 0.0001) they think about the HIRAID ® steps while doing a patient assessment and that HIRAID ® helped them in clinical practice (85.6%, n = 137, p < 0.0001) (Supplementary File 3).There were four themes generated from 721 codes provided in participant free text comments explaining why they did or did not feel HIRAID ® supported patient assessment, particularly those new to emergency nursing as a tool that (i) Prompts and provides structure for consistency (ii) enabled more consistent identification of potential problems and communication, (iii) impacts learning, education and training needs; and (iv) impacts current work practice (Table 5).

Adoption
Adoption of the intervention was extremely high at the setting and individual levels.The overall site adoption of the intervention was 100%.At least one Instructor from each site attended Instructor training.
At the individual level, overall adoption was high as evidenced by nurse use of the HIRAID ® documentation templates.Six weeks post implementation, three initial consecutive nursing entry records of every nurse on shift (n = 711) over a 7-day period were audited, 83% demonstrated appropriate use of the HIRAID ® documentation templates when indicated at least once.This proportion remained consistent at 12 weeks, 84% of 989 nurses appropriately used the templates (Fig. 2).Uptake at the metropolitan sites was much higher than the rural regional sites (95% vs 76%).Of nurses who completed the survey six months post implementation, 88.6% (n = 473) indicated they use the HIRAID ® documentation templates most or all the time for their initial documentation.This ranged from 82.5% at one of the rural clusters to 92.6% at the other rural cluster.
There were four themes generated from 149 comments provided by survey respondents in response to questions asking why they did or did not use the HIRAID ® documentation templates.The most frequent reason was that a standardised approach with prompts enabled more comprehensive care, consistency of documentation, and reduced cognitive load (Table 5).

Implementation quality: Dose
Of the 21 BCTs, 100% were used.Dose was high, as assessed through the number and proportion of instructor and provider courses delivered at each cluster and site within the proposed timeline.At a setting level, there were 15 Instructor courses delivered at 11 sites by one of the HIRAID ® Chief investigators across all clusters with staff from all sites (100% dose).Then, these Instructors delivered 210 provider courses at 90% (26/29) of sites within the study protocol timeline and study budget.Feedback from 104 of the 145 Instructor course attendees was that the learning outcomes were clearly presented mean (SD) (8.7 (± 1.6), and the course was engaging 9.2 (± 1.4).Respondents indicated they were clear on how to apply the HIRAID ® framework in clinical practice 9 (± 1.4) and had a good understanding of their role as a HIRAID ® Instructor 8.9 (± 1.4) and felt confident in their ability to deliver the training 8.8 (± 1.3).These high levels of confidence and understanding were consistent across sites and supported by more than 60 free text comments (Table 6).

Implementation quality: fidelity
All 21 implemented BCTTs were implemented as intended, three with minor modification and one with moderate modification.The instructor courses (setting level) were delivered with extremely high fidelity.
The same cohort of four instructors delivered these and other than the planned site-specific modifications, for example, the names of the hospitals or nursing executive, the content was unchanged.The mode of delivery did vary for one Instructor course as two days before training was to begin in one cluster, catastrophic flooding occurred meaning flights to the venue were cancelled, roads were closed, and several staff lost their homes.This required some modification to how the course was delivered (from all face to face, to hybrid face to face / online), but not the content.There was also some variation to the mode of delivery to some participant courses in the same cluster as the second wave of the COVID-19 pandemic was in its peak.Smaller group sizes, mask wearing and a brief move from all face to face, to hybrid face to face / online courses was necessary (Table 6).No additional funding was required to complete implementation.
HIRAID ® champions (self-identified staff to feedback and support in practice) were implemented at all sites however their capacity to complete the intended tasks was moderate (49% all the time, 47% sometimes) and varied across clusters (27% at one metro cluster were able to complete intended tasks all the time compared to 67% at the other metro cluster).Instructors felt they were able to explain what HIRAID ® was most of the time (98%) but only 60% felt they always had time and 35% sometimes had time to go through documentation templates with staff.Even fewer felt they had time to work with novice nurses to complete a patient assessment using HIRAID ® (49% all the time, 35% sometimes) or practice handover and escalation with novice nurses (32% all the time, 46% sometimes).The HIRAID ® teaching materials were delivered by those who had attended instructor training 94% of the time and when the provided slide deck was used (67%), it was used as intended (87.8%).The only changes made were to suit local context, for example, inserting local statistics.The rural clusters used the slides about half the time education was delivered (Supplementary file 4).

Discussion
The implementation of a behaviour-change informed strategy for the emergency nursing framework HIRAID ® in rural, regional, and metropolitan Australia was highly successful, achieving 100% adoption and high reach, effectiveness, adoption dose, fidelity, and sustainability.Guided by the RE-AIM framework [23], we used multiple methods to demonstrate that all 21 BCT and mechanisms as determined by the behaviour change wheel process were used to high fidelity, some requiring minor modification for circumstances beyond our control such as the COVID-19 pandemic and catastrophic flooding.The use of behaviour change theory enabled maximum uptake of our intervention and adherence to intervention fidelity in the complex ED clinical settings where others have failed [10].Implementation in emergency contexts is challenging with numerous barriers, with high staff turnover, overcrowding and high priorities.High staff turnover often means there are high numbers of non-permanent staff.Non-permanent staff were not included in the evaluation; however, are an essential part of nursing workforce and need consideration.This has since occurred through generation of "cheat sheets" for how to use HIRAID ® and local rapid onboarding processes.The implementation plan included strategies for maintenance of the use of HIRAID ® embedded into local roles and practices such as education.
Challenges related to COVID-19 and catastrophic flooding added challenges to implementation.Implementing a major project during a pandemic and a natural disaster meant flexibility in design, delivery and expected progress.Adapting to challenges was critical to progress.Though some of the education sessions had to be adapted to online, our experience was that virtual in-services were well attended, and engagement was not compromised.Adapting to challenges while maintaining high fidelity is possible in the ED [46]; however the success or failure of these adaptations is less certain [47].Using implementation science frameworks to be adaptive to the needs of the sites during implementation and having strong champions and site engagement has been shown to be successful [48].Our experience was that relationships and effective regular communication with stakeholders were critical components to problem solve and meet local needs, cognisant of impact on personal impacts on staff related to the major events.As the implementation was part of a large funded study, there was access to highly experienced implementation specialists and expert emergency clinician researchers, access to specialists and funding to enable expertise and support would likely contribute to the success of implementation [49].
Our evaluation adds to and challenges the work of Connell et al. [50] around the four of the ten BCTs and Mechanisms of Action reported by a panel of 105 experts as having no definite links.In developing our implementation strategy, we allocated the Mechanisms of Action "Optimism" and "Social Professional Role and Identity" to address barriers related to feelings of a lack of support by management and a lack of time to change practice.Optimism, that is, the confidence that things will happen for the best or that desired goals are attained was effectively linked to two BCTs in this study: (i) 6.3 Information about others approval was demonstrated through leadership support for implementation, e.g.memo from hospital executives to nursing staff supporting the project, and commitment of work time to complete education.In addition, leadership approval was demonstrated in a video for viewing in the training program featured senior staff and relevant team members.(ii) 6.2 Social Comparison was used to reassure staff about the change and provide evidence to staff how implementation was achieved at other sites in all training.
"Social Professional Role and Identity" was linked to three BCTs (i) 6.1 demonstration of the behaviour by having instructors from other sites attend train the trainer sessions at other clusters demonstrating and modelling behaviour (ii)13.2Framing / reframing to ensure experienced staff understand that the HIRAID ® intervention does not necessarily change work practice but provides common terminology and approach to assessment in the ED.This was incorporated throughout the education program and, (iii) 15.1 Verbal persuasion about capability.Instructors developed phrases to support/ encourage staff use of HIRAID ® -delivered and codesigned in the Instructor education program that were then used as part of implementation.
Two BCTs Action Planning (1.4) and Credible Source (9.1) were linked to three mechanisms of action.Action Planning (1.4) was successfully linked to the mechanisms of action "Belief about consequences" to address the barrier "a lack of belief that change will happen".Action planning was used to operationalise an implementation plan for each site, including who is making the changes, timeframes, milestones and progress measures.This demonstrated to staff that there was accountability and expectation that change would occur."Credible Source (9.1)" was linked to the mechanisms of action Goals to facilitate the enabler "evidence that implementation of the intervention will help do what is best for patient care" and Social Influences to address the barrier "lack of support" through an email from executive outline their support of HIRAID ® as evidenced by allocation of nonclinical time to attend training.

Research and practical implications
The generation of robust evidence for implementation strategy effectiveness and measurement is in its infancy, particularly in the emergency setting [23], however, this research has advanced knowledge in this field, for example, by highlighting the importance of social influence, professional identity, reframing and verbal persuasion [51].These were achieved effectively in this study using site champions [52]; however, would unlikely be effective as a strategy alone [53].Targeted contextual barrier assessment and identification of relevant strategies were successful in this implementation study, which may be the key until we have further evidence on specific strategies.Planning without use of implementation science frameworks is not adequate in the emergency context [54] and consideration to enabling and engaging front line clinicians is crucial.
In this study, we used the RE-AIM framework to guide the evaluation of the implementation, finding it appropriate for our needs in this study.The evaluation plan was developed prior to implementation of the study and developed benchmarks; however, some benchmarks were arbitrarily selected as others have identified [55].There is limited evidence in the implementation fidelity literature for setting benchmarks, much of which focuses on adaptations.Implementation research guidelines on levels of fidelity, adaptations and sustainability in the acute care and emergency setting should be developed to inform future research.
This study provided evidence that a novel framework for emergency nursing can be adapted to context and delivered as intended yielding a high reach.The results of this study informed changes to the implementation plan and the development of an implementation toolkit to support implementation spread and scale-up at new sites wanting to implement HIRAID ® .Since implementation at the study sites, HIRAID ® has been implemented using our implementation plan at more than 50 EDs in Australia, and to varying degrees in Fiji, Canada, USA, China and Singapore through their emergency nursing associations (see Supplementary material 5).We are working our study partners, the Australian College of Nursing and the Australian Commission for Quality and Safety in Health care to further upscale locally.

Methodological considerations and limitations
There were some limitations to this study.Our sites represented rural, regional, and metropolitan settings however may still not be generalisable to all contexts.Our implementation strategy should not be used without examination and consideration for the proposed context.Documentation audits may not give a true indication of staff members using HIRAID ® framework in their assessments.The surveys were developed based on responses needed to address the RE-AIM criteria, however, there were no standardised questions available.The qualitative questions were limited to questions about the use of the tool rather than evaluation of the process, which may limit the interpretation of some of the results.There was some delay between the implementation and the surveys which may have resulted in participants misremembering some answers.We did not conduct documentation audits for non-permanent staff as they were less frequently on shift, allocated shifts through different rostering systems and not formally part of implementation however their contribution to emergency care is crucial and their training and education is important to consider in further studies in emergency contexts.The next phase of this program of research is to evaluate the impact of the implementation of HIRAID ® on patient and health service outcomes, including cost-benefit to determine the economic soundness of the intervention.

Conclusion
The implementation of a behaviour-change informed strategy for the emergency nursing framework HIRAID ® in rural, regional, and metropolitan Australia was highly successful, achieving 100% setting adoption with high reach, dose, fidelity, and sustainability up to 3 years post implementation.The use of behaviour change theory enabled maximum uptake of our intervention and adherence to intervention fidelity in complex ED clinical settings.We made some modifications to the original implementation strategy in response to major incidents.The project was delivered within the study protocol timeline and study budget.Future upscaling can be informed by this evidenced-based, reliable strategy.

Table 1
RE-AIM evaluation plan to evaluate HIRAID ®

Table 2
Trial Cluster, ED patient presentations, admissions, nursing staff and trial start date Abbreviations: ED Emergency departments, km 2 Kilometre, LHD Local health district

Table 3
Post HIRAID ® implementation nursing survey respondent characteristics overall and by cluster

Table 4
Setting and individual reach of the HIRAID ® implementation education program

Table 5
Content analysis of nursing survey responses around HIRAID ® use, subthemes and exemplar quotes

Why do or don't you use the HIRAID documentation templates? 149 Codes
really need is safe staffing and skill mix so that senior nurses have the time to model and teach excellent assessment, and educators that work and teach "at the elbow" where the learning happens" (Participant 358, Cluster 2) Not appropriate for the clinical situation 19 ▪ "If in resus, to many things changing quickly so a note works better.Hi raid can then come later" (Participant 572, Cluster 4) ▪ "not every presentation needs an assessment eg. a splinter in a finger." (Participant 102 Cluster 1) Prioritising direct clinical care 33 Workload and patient acuity 22 Not enough staff 11 ▪ "We don't have the time! 1 nurse and up to 10 patients It' s often IMPOSSIBLE" (Participant 47 Cluster 1) Why does/does not HIRAID ® support

nurses to assess and manage ED patients 721 Codes
▪ "HIRAID doesn't leave room for forgetting / missing parts of an assessment.It is very busy in ED and sometimes I will forget to assess something in my re-assessment of my patient, and when I begin documenting, the HIRAID template reminds me to go back to the patient and assess what I missed" (participant 149, Cluster 3) ▪ "It does prompt me to ask questions of the patient when I am documenting, especially when it is busy, or the patient is very complex, and I am documenting in retrospect" (participant 67, Cluster 1) ▪ "HIRAID in a way breaks down doing an assessment.I know I initially got overwhelmed with trying to gather information, but when HIRAID was introduced it simplified it for me" (participant 175, Cluster 4) template to help junior nurses organise their notes and hopefully direct their care, but the template alone is not going to change practice.What we really need is safe staffing and skill mix so that senior nurses have the time to model and teach excellent assessment, and educators that work and teach "at the elbow" where the learning happens" (Participant 358, Cluster 2) ▪ "I agree the HIRAID does help junior nurses to think about their assessment and interventions.I don't think it "teaches"-that is the job of the educator and other more senior nurses" (Participant 324, Cluster 2) Impacts current work practice 98

Table 5 (continued) Why do or don't you use the HIRAID documentation templates? 149 Codes
Fig. 2 Proportion of nurses using HIRAID templates at 6, 12, 24 weeks post implementation.*24 week data from Cluster 2 not available

Table 6
Behaviour change techniques, mode of delivery and use in HIRAID ®

Table 6
29ontinued)At a setting level, at the time of publication (3 years since implementation),29EDs have embedded the HIRAID ® training program in their new staff orientation program and recommend use of HIRAID ® to inform initial patient assessment and management post triage.This is evidenced by correspondence from site educators with whom we have an ongoing relationship.Since completion of the study, site educators track completion of the HIRAID ® program by new staff in local learning tracking software used by each health district for nursing education.Updated training materials, such as revisions to workbooks or readings are distributed to the educators and leadership teams for each site.Electronic documentation templates also remain in use, with the ability for each site to modify for context if needed.eLearning site has been updated and we are able to identify the location of staff who have completed the training.At an individual level, 94.8% (n = 506) of respondents to the 6-month post implementation survey indicated they used a preplanned structure to clinical assess their patient and 88.6% (n = 473) said they were using the HIRAID ® documentation templates to record this.This was supported by 95% (n = 621) of nurses at three clusters demonstrating appropriate use of the templates at least once in a 6-month documentation audit.